Numerical simulations of turbulent Rayleigh-B\'enard convection in an ideal gas, using either the anelastic approximation or the fully compressible equations, are compared. Theoretically, the anelastic approximation is expected to hold in weakly superadiabatic systems with $\epsilon = \Delta T / T_r \ll 1$, where $\Delta T$ denotes the superadiabatic temperature drop over the convective layer and $T_r$ the bottom temperature. Using direct numerical simulations, a systematic comparison of anelastic and fully compressible convection is carried out. With decreasing superadiabaticity $\epsilon$, the fully compressible results are found to converge linearly to the anelastic solution with larger density contrasts generally improving the match. We conclude that in many solar and planetary applications, where the superadiabaticity is expected to be vanishingly small, results obtained with the anelastic approximation are in fact more accurate than fully compressible computations, which typically fail to reach small $\epsilon$ for numerical reasons. On the other hand, if the astrophysical system studied contains $\epsilon\sim O(1)$ regions, such as the solar photosphere, fully compressible simulations have the advantage of capturing the full physics. Interestingly, even in weakly superadiabatic regions, like the bulk of the solar convection zone, the errors introduced by using artificially large values for $\epsilon$ for efficiency reasons remain moderate. If quantitative errors of the order of $10\%$ are acceptable in such low $\epsilon$ regions, our work suggests that fully compressible simulations can indeed be computationally more efficient than their anelastic counterparts.